Processing system for data elements received via source inputs

ABSTRACT

Mediums, apparatus, computer program code, and means may be provided to evaluate relative risks based at least in part on source inputs received via a distributed communication network by an automated back-end application computer server. According to some embodiments, the server may access a data store containing electronic files associated with a set of entities to retrieve, for each of a plurality of the entities in the set of entities, electronic files associated with that entity. The server may also retrieve structured data elements, unstructured data elements, and external, third-party data elements for that entity. The server may then execute an automated risk model to assign a risk score to that entity based on the electronic files, the structured data elements, the unstructured data elements, and the external, third-party data elements for that entity and transmit indications of the risk scores for the plurality of entities.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of U.S. Provisional PatentApplication No. 62/261,059 entitled “PROCESSING SYSTEM FOR DATA ELEMENTSRECEIVED VIA SOURCE INPUTS” and filed Nov. 30, 2015. The entire contentof that application is incorporated herein by reference.

BACKGROUND

A set of entities may include different entities that are eachassociated with a different level of risk. For example, a first entitymay pose a first degree of risk that is substantially higher as comparedto a second entity that is associated with a second degree of risk. Itmay therefore be desirable to determine relative amounts of risk foreach entity in a set. For example, determining that a particular entityis associated with an unusually high level of risk may allow remedialactions to be taken with respect to that entity. Determining thisinformation, however, can be a time consuming and error prone task,especially where then are a substantially number of entities and whendegrees of risk may depend on data elements from various source inputsof different types (e.g., structured source inputs, unstructured sourceinputs, third-party source inputs, etc.).

It would be desirable to provide systems and methods to process dataelements received via source inputs in a way that provides faster, moreaccurate results and that allows for flexibility and effectiveness whenresponding to those results.

SUMMARY OF THE INVENTION

According to some embodiments, systems, methods, apparatus, computerprogram code and means are provided to process data elements receivedvia source inputs in a way that provides faster, more accurate resultsand that allows for flexibility and effectiveness when responding tothose results. A back-end application computer server may access a datastore containing electronic files associated with a set of entities toretrieve, for each of a plurality of the entities in the set ofentities, electronic files associated with that entity. The server mayalso retrieve structured data elements, unstructured data elements, andexternal, third-party data elements for that entity. The server may thenexecute an automated risk model to assign a risk score to that entitybased on the electronic files, the structured data elements, theunstructured data elements, and the external, third-party data elementsfor that entity and transmit indications of the risk scores for theplurality of entities.

Some embodiments comprise: means for accessing a data store containingelectronic files associated with a set of entities to retrieve, for eachof a plurality of the entities in the set of entities, electronic filesassociated with that entity; means for retrieving structured dataelements for that entity from a structured data element informationsource input; means for retrieving unstructured data elements for thatentity from an unstructured data element information source input; meansfor retrieving external, third-party data elements for that entity fromthe external, third-party data element information source input; meansfor executing, by a computer processor of the back-end applicationcomputer server, an automated risk model to assign a risk score to thatentity based on the electronic files, the structured data elements, theunstructured data elements, and the external, third-party data elementsfor that entity; and means for transmitting indications of the riskscores for the plurality of entities.

In some embodiments, a communication device associated with a back-endapplication computer server exchanges information with remote devices.The information may be exchanged, for example, via public and/orproprietary communication networks.

A technical effect of some embodiments of the invention is an improvedand computerized ways to process data elements received via sourceinputs to provide faster, more accurate results and that allow forflexibility and effectiveness when responding to those results. Withthese and other advantages and features that will become hereinafterapparent, a more complete understanding of the nature of the inventioncan be obtained by referring to the following detailed description andto the drawings appended hereto.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is block diagram of a system according to some embodiments of thepresent invention.

FIG. 2 illustrates a method according to some embodiments of the presentinvention.

FIG. 3 is block diagram of a system in accordance with some embodimentsof the present invention.

FIGS. 4 through 8 illustrate exemplary displays that might be associatedwith various embodiments described herein.

FIG. 9 is a block diagram of an apparatus in accordance with someembodiments of the present invention.

FIG. 10 is a portion of a tabular database storing risk score results inaccordance with some embodiments.

FIG. 11 illustrates a system having a predictive model in accordancewith some embodiments.

FIG. 12 is an example of a book of business view according to someembodiments.

FIG. 13 illustrates a new variable proving ground in accordance withsome embodiments.

FIG. 14 is associated with a large workers' compensation loss on anout-of-appetite risk according to some embodiments.

FIG. 15 illustrates a period execution of a risk review implementationaccording to some embodiments.

FIG. 16 illustrates a tablet computer displaying risk score dashboardinformation according to some embodiments.

FIG. 17 is an example of a book of business display of potentialresponses that might be presented in accordance with some embodiments.

DETAILED DESCRIPTION

The present invention provides significant technical improvements tofacilitate dynamic data processing. The present invention is directed tomore than merely a computer implementation of a routine or conventionalactivity previously known in the industry as it significantly advancesthe technical efficiency, access and/or accuracy of communicationsbetween devices by implementing a specific new method and system asdefined herein. The present invention is a specific advancement in thearea of evaluating risk based on data element source inputs by providingtechnical benefits in data accuracy, data availability and dataintegrity and such advances are not merely a longstanding commercialpractice. The present invention provides improvement beyond a meregeneric computer implementation as it involves the processing andconversion of significant amounts of data in a new beneficial manner aswell as the interaction of a variety of specialized client and/or thirdparty systems, networks and subsystems. For example, in the presentinvention information may be transmitted from remote devices to aback-end application server and then analyzed accurately to evaluatelevels of risk to improve the overall performance of the system.

Note that, in a computer system, different entities may each beassociated with a different level of risk. For example, a first entitymay pose a first degree of risk that is substantially higher as comparedto a second entity that is associated with a second degree of risk. Itmay therefore be desirable to determine relative amounts of risk foreach entity in a set. For example, determining that a particular entityis associated with an unusually high level of risk may allow remedialactions to be taken with respect to that entity. Determining thisinformation, however, can be a time consuming and error prone task,especially where then are a substantially number of entities and whendegrees of risk may depend on data elements from various source inputsof different types (e.g., structured source inputs, unstructured sourceinputs, third-party source inputs, etc.). It would be desirable toprovide systems and methods to process data elements received via sourceinputs in a way that provides faster, more accurate results and thatallows for flexibility and effectiveness when responding to thoseresults. FIG. 1 is block diagram of a system 100 according to someembodiments of the present invention. In particular, the system 100includes a back-end application computer server 150 that may accessinformation in a computer store 110. The back-end application computerserver 150 may also exchange information with a remote administratorcomputer 160 (e.g., via a firewall 120) and/or source inputs 142, 144,146. According to some embodiments, a risk model 130 of the back-endapplication computer server 150 may facilitate the display of riskevaluation information via one or more remote administrator computers160.

The back-end application computer server 150 might be, for example,associated with a Personal Computer (“PC”), laptop computer, smartphone,an enterprise server, a server farm, and/or a database or similarstorage devices. According to some embodiments, an “automated” back-endapplication computer server 150 may facilitate the evaluation of riskassociated with entities in the computer store 110. As used herein, theterm “automated” may refer to, for example, actions that can beperformed with little (or no) intervention by a human.

As used herein, devices, including those associated with the back-endapplication computer server 150 and any other device described hereinmay exchange information via any communication network which may be oneor more of a Local Area Network (“LAN”), a Metropolitan Area Network(“MAN”), a Wide Area Network (“WAN”), a proprietary network, a PublicSwitched Telephone Network (“PSTN”), a Wireless Application Protocol(“WAP”) network, a Bluetooth network, a wireless LAN network, and/or anInternet Protocol (“IP”) network such as the Internet, an intranet, oran extranet. Note that any devices described herein may communicate viaone or more such communication networks.

The back-end application computer server 150 may store information intoand/or retrieve information from the computer store 110. The computerstore 110 might, for example, store data associated with a set ofentities, each entity being associated with a different level of risk.The computer store 110 may also contain past and current interactionswith source inputs 142, 144, 146 on historic entities. The computerstore 110 may be locally stored or reside remote from the back-endapplication computer server 150. As will be described further below, thecomputer store 110 may be used by the back-end application computerserver 150 to generate and/or calculate risk parameters that will betransmitted to the remote administrator computer 160. Although a singleback-end application computer server 150 is shown in FIG. 1, any numberof such devices may be included. Moreover, various devices describedherein might be combined according to embodiments of the presentinvention. For example, in some embodiments, the back-end applicationcomputer server 150 and computer store 110 might be co-located and/ormay comprise a single apparatus.

According to some embodiments, the system 100 may evaluate riskinformation over a distributed communication network via the automatedback-end application computer server 150. For example, at (1) the remoteadministrator computer 160 may request that an automated risk analysisbe performed. The back-end application computer server 150 may thendetermine information about a set of entities in the computer store 110at (2) and collect data associated with the source inputs 142, 144, 146at (3). In particular, data might be collected from a structured dataelement source input 142, an unstructured data element source input 144,and/or an external, third-party data element source input 146. The riskmodel 130 may then be executed and results may be provided to the remoteadministrator computer 160 at (4).

Note that the system 100 of FIG. 1 is provided only as an example, andembodiments may be associated with additional elements or components.According to some embodiments, the elements of the system 100 evaluaterisk over a distributed communication network. FIG. 2 illustrates amethod 200 that might be performed by some or all of the elements of thesystem 100 described with respect to FIG. 1, or any other system,according to some embodiments of the present invention. The flow chartsdescribed herein do not imply a fixed order to the steps, andembodiments of the present invention may be practiced in any order thatis practicable. Note that any of the methods described herein may beperformed by hardware, software, or any combination of these approaches.For example, a computer-readable storage medium may store thereoninstructions that when executed by a machine result in performanceaccording to any of the embodiments described herein.

At S210, a data store containing electronic files associated with a setof entities may be accessed to retrieve, for each of a plurality ofentities in the set of entities, electronic files associated with thatentity. According to some embodiments, the data store is associated withan insurer's book of business, and each entity in the data storecomprises an insured party. Other embodiments might be associated withnewly submitted policy requests, policy renewals, underwriting andpricing decisions, etc. Note that an insurer's book of business maycover or more types of insurance, including workers' compensationinsurance, commercial automobile insurance, general liability insurance,property insurance, property casualty insurance, homeowners' insurance,personal lines of insurance, group benefits insurance, and/or lifeinsurance. In some cases, not all of the entities in the computer storemay be evaluated. For example, the plurality of insured parties mayrepresent insured parties that do not satisfy a pre-determined thresholdcriteria indicating that a substantial potential loss is unlikely.

At S220, structured data elements may be retrieved for that entity froma structured data element information source input. The structured dataelements might be, for example, associated with internally stored dataelements about the insured party (e.g., claim notes store by theinsurer) and/or governmental or regulatory data elements (e.g.,indicating that an entity did, or did not, pass an inspection test).

At S230, unstructured data elements may be retrieved for that entityfrom an unstructured data element information source input. Theunstructured data elements might be, for example, associated with webpages of the insured party (e.g., where the entity describes theoperation of the business) and/or web pages mentioning the insured party(e.g., providing reviews of the goods or services provided by theentity).

At S240, external, third-party data elements may be retrieved for thatentity from an external, third-party data element information sourceinput. The external, third-party data elements might be, for example,associated with social media content (e.g., where people exchange textand images with each other).

At S250, a computer processor of a back-end application computer servermay execute an automated risk model to assign a risk score to thatentity based on the electronic files, the structured data elements, theunstructured data elements, and the external, third-party data elementsfor that entity. Note that the values associated with any of these datasources might comprise flags, set of flags, scores along a spectrum ofrisk, buckets collecting multiple values for a single account, etc. Therisk score may, for example, represent a likelihood of a substantialpotential loss for the insurer. The automated risk model may beexecuted, for example: when initiated by an administrator, responsive toa specific query submitted by the administrator (who might ask, forexample, to see all risk information associated with drones), on aperiodic basis, on a daily basis, on a weekly basis, on a monthly basis,and/or on a substantially continuous basis. In the case of risk modelexecution on a substantially continuous basis, the system may constantlyextract information from relevant web pages, etc., in substantially realtime and perform some degree of pre-processing on the data. In somecases, the results of this pre-processing might result in a warning flagor message being automatically routed to the administrator or a manager.According to some embodiments, prior to execution at t S250, the systemmay determine at least one variable associated with the automated riskmodel, the at least one variable being assigned a predictive qualitybased at least in part on an input frequency of the variable (e.g., asdescribed with respect to FIG. 13). In this case, execution of the modelmay further be based at least in part on the automatically identifiedvariable.

At S260, indications of the risk scores for the plurality of entitiesmay be transmitted. According to some embodiments, the risk scorecomprises a numerical value for each insured party. According to otherembodiments, the risk score is associated with a ranked list of insuredparties. Note that a risk score might indicate, for example, that apolicy will not be issued, that a policy will be issued, or that amanager having a particular level of experience or area of expertisewill need to approve the insurance policy.

Some of the embodiments described herein may be implemented via aninsurance enterprise system. For example, FIG. 3 is block diagram of asystem 300 according to some embodiments of the present invention. As inFIG. 1, the system 300 includes a back-end application computer server350 that may access information in a computer store 310. The back-endapplication computer server 350 may also exchange information with aremote administrator computer, a schedule task manager 360 (e.g., amanager 360 that automatically requests risk information on a weeklybasis via a firewall 320), and/or source inputs 342, 344, 346, 348.According to some embodiments, a risk of substantial loss predictivemodel 330 and text mining tool 332 of the back-end application computerserver 350 facilitates the transmission of risk information to theschedule task manager 360.

The back-end application computer server 350 might be, for example,associated with a PC, laptop computer, smartphone, an enterprise server,a server farm, and/or a database or similar storage devices. Theback-end application computer server 350 may store information intoand/or retrieve information from the computer store 310. The computerstore 310 might, for example, store data associated with past andcurrent insurance policies. The computer store 310 may be locally storedor reside remote from the back-end application computer server 350. Aswill be described further below, the computer store 310 may be used bythe back-end application computer server 350 to generate and/orcalculate risk parameters.

According to some embodiments, the system 300 may evaluate riskinformation over a distributed communication network via the automatedback-end application computer server 350. For example, at (1) the remoteschedule task manager 360 may request that an automated risk analysis beperformed. The back-end application computer server may then determineinformation about a set of entities in the computer store 310 at (2). Inparticular, the entities may comprise insured parties in an insurer'sbook of business. Consider, for example, FIG. 4 which illustrates adisplay 400 of an insurer's book of business for a particular type ofinsurance (e.g., in this example, or the workers' compensation line ofbusiness). The display 400 may include a search function 410 to let anadministrator locate a particular insured. The display 400 furtherincludes details 420 about the insurer's book of business, including anaccount identifier, an account name, an insurance policy number, and aprior risk score for each account in the book. The display 400 mayfurther include an icon 430 that may be selected by an administrator torequest a risk report.

Referring again to FIG. 3, he back-end application server 350 may thencollect data associated with the source inputs 342, 344, 346, 348 at(3). In particular, data might be collected from a structured dataelement source input, such as governmental Occupational Safety andHealth Administration (“OSHA”) reports 342. Consider, for example, FIG.5 which illustrates an OSHA violation web page display 500 or datasource. Note that the data elements 510 in the display 500 may be“structured” in that various elements (e.g., a violation identifier, acompany name, a date of violation, and/or a violation category) may beknown to be located at pre-determined positions or locations within thedisplay 500. In this way, the system may automatically match the companyname of an insured (e.g., “Smith Industries”) and use the associatedviolation information 520, including the violation category, whenexecuting the risk model.

Referring again to FIG. 3, the back-end application computer server 350may also collect information from an unstructured data element sourceinput, such as one or more web pages run by the insured 344. Consider,for example, FIG. 6A which illustrates a web page display 600 maintainedby an insured (Acme Landscaping). Note that the data elements in thedisplay 600 may be “unstructured” in that various elements (e.g., textand images) may not be located at pre-determined positions or locationswithin the display 600. Instead, the system may automatically search forand discover risk related information (e.g., the fact that the companyhas a newly offered “tree trimming service”) and use the detectedinformation 610 when executing the risk model. Further note that thephrase “data element,” as used herein, might refer to different types ofinformation, including text information, image information (photographsand moving images), sound information (including speech and other typesof audio information), etc. For example, FIG. 6B illustrates a display602 including unstructured data elements 612, 622 other than textinformation. The system may, for example, examine the unstructured dataelements 612, 622 looking for relevant risk information (e.g., the useof a particular type of construction tool, the lack of safety equipment,etc.). Similarly, FIG. 6C illustrates a display 604 with data elements614, 624 that have been collected by an insurance enterprise (e.g., viadrone or satellite images). The system may again examine theunstructured data elements 614, 624 looking for relevant riskinformation (e.g., work on a particular type of building that isconsidered especially dangerous).

Referring again to FIG. 3, the back-end application computer server 350may also collect information from other public records 346 (e.g., courtdocuments, police records, etc.). According to some embodiments, theback-end application computer server 350 may also collect informationfrom an external, third-party data element source input, such as socialmedia content 348. Consider, for example, FIG. 7 which illustrates asocial media site display 700 wherein people may post text and imageinformation 710 about an establishment. Note that the data elements inthe display 700 may be “unstructured” in that various elements (e.g.,text and images) may not be located at pre-determined positions orlocations within the display 700. Instead, the system may automaticallysearch for and discover risk related information (e.g., the fact thatthe company has recently been granted a liquor license to sell alcohol)and use the detected information 720 when executing the risk model. Notethat information from product and/or service review sites might be usedinstead of, or in addition to, social media content.

Referring again to FIG. 3, the risk model 330 may then be executed andresults may be provided to the schedule task manager at (4). Consider,for example, FIG. 8 which illustrates a risk score result display 800for an insurer's book of business. Note that the risk score results 810might include a numerical value for each account (e.g., a current riskscore of 80, 60, 45, etc.) and/or a ranked list of accounts (fromhighest-to-lowest risk or lowest-to-highest risk).

The embodiments described herein may be implemented using any number ofdifferent hardware configurations. For example, FIG. 9 illustrates aback-end application computer server 900 that may be, for example,associated with the systems 100, 300 of FIGS. 1 and 3, respectively. Theback-end application computer server 900 comprises a processor 910, suchas one or more commercially available Central Processing Units (“CPUs”)in the form of one-chip microprocessors, coupled to a communicationdevice 920 configured to communicate via a communication network (notshown in FIG. 9). The communication device 920 may be used tocommunicate, for example, with one or more remote administratorcomputers. Note that communications exchanged via the communicationdevice 920 may utilize security features, such as those between a publicinterne user and an internal network of the insurance enterprise. Thesecurity features might be associated with, for example, web servers,firewalls, and/or PCI infrastructure. The back-end application computerserver 900 further includes an input device 940 (e.g., a mouse and/orkeyboard to enter information about risk scoring rules or businesslogic, historic information, predictive models, etc.) and an outputdevice 950 (e.g., to output reports regarding system administration,risk recommendations, and/or insured parties).

The processor 910 also communicates with a storage device 930. Thestorage device 930 may comprise any appropriate information storagedevice, including combinations of magnetic storage devices (e.g., a harddisk drive), optical storage devices, mobile telephones, and/orsemiconductor memory devices. The storage device 930 stores a program915 and/or a risk evaluation tool or application for controlling theprocessor 910. The processor 910 performs instructions of the program915, and thereby operates in accordance with any of the embodimentsdescribed herein. For example, the processor 910 may access a data storecontaining electronic files associated with a set of entities toretrieve, for each of a plurality of the entities in the set ofentities, electronic files associated with that entity. The processor910 may also retrieve structured data elements, unstructured dataelements, and/or external, third-party data elements for that entity.The processor 910 may then execute an automated risk model to assign arisk score to that entity based on the electronic files, the structureddata elements, the unstructured data elements, and the external,third-party data elements for that entity and transmit indications ofthe risk scores for the plurality of entities.

The program 915 may be stored in a compressed, uncompiled and/orencrypted format. The program 915 may furthermore include other programelements, such as an operating system, a database management system,and/or device drivers used by the processor 910 to interface withperipheral devices.

As used herein, information may be “received” by or “transmitted” to,for example: (i) the back-end application computer server 900 fromanother device; or (ii) a software application or module within theback-end application computer server 900 from another softwareapplication, module, or any other source.

In some embodiments (such as shown in FIG. 9), the storage device 930further stores a computer store 960 (e.g., associated with a book ofbusiness and past policy submissions, underwriting decisions, premiums,claims, damages, etc.) and a risk score results database 1000. Anexample of a database that might be used in connection with the back-endapplication computer server 900 will now be described in detail withrespect to FIG. 10. Note that the database described herein is only anexample, and additional and/or different information may be storedtherein. Moreover, various databases might be split or combined inaccordance with any of the embodiments described herein. For example,the computer store 960 and/or risk score results database 1000 might becombined and/or linked to each other within the program 915.

Referring to FIG. 10, a table is shown that represents the risk scoreresults database 1000 that may be stored at the back-end applicationcomputer server 900 according to some embodiments. The table mayinclude, for example, entries identifying accounts in an insurer's bookof business. The table may also define fields 1002, 1004, 1006, 1008,1010, 1012 for each of the entries. The fields 1002, 1004, 1006, 1008,1010, 1012 may, according to some embodiments, specify: an accountidentifier 1002, an account name 1004, an insurance policy number 1006,an insurance type 1008, source input data elements 1010, and a riskscore 1012. The risk score results database 1000 may be created andupdated, for example, based on information electrically received from acomputer store and one or more source inputs.

The account identifier 1002 may be, for example, a unique alphanumericcode identifying an insured party, and the account name 1004 and theinsurance policy number 1006 may be associated with that party. Theinsurance type 1008 may be used to define an insurer's book of business(e.g., for workers'compensation, commercial automobile, etc.). Thesource input data elements 1010 may represent, for example, informationthat was found from structured source, unstructured sources, socialmedia content, etc. that may increase (or decrease) the likelihood thatthe account may cause a substantial loss to the insurer. The risk score1012 might represent numeric value, category (red, yellow, green), anoverall ranking, etc., representing an amount of risk associated withthe account identifier 1002.

According to some embodiments, one or more predictive models may be usedto select and/or score risk information (e.g., the risk score 1012 inthe risk score results database 1000). Features of some embodimentsassociated with a predictive model will now be described by firstreferring to FIG. 11. FIG. 11 is a partially functional block diagramthat illustrates aspects of a computer system 1100 provided inaccordance with some embodiments of the invention. For present purposesit will be assumed that the computer system 1100 is operated by aninsurance company (not separately shown) for the purpose of supportingautomated insurance agency evaluations (e.g., assigning risk scores toaccounts in an insurer's book of business).

The computer system 1100 includes a data storage module 1102. In termsof its hardware the data storage module 1102 may be conventional, andmay be composed, for example, by one or more magnetic hard disk drives.A function performed by the data storage module 1102 in the computersystem 1100 is to receive, store and provide access to both historicaltransaction data (reference numeral 1104) and current transaction data(reference numeral 1106). As described in more detail below, thehistorical transaction data 1104 is employed to train a predictive modelto provide an output that indicates an identified performance metricand/or an algorithm to score risk factors, and the current transactiondata 1106 is thereafter analyzed by the predictive model. Moreover, astime goes by, and results become known from processing currenttransactions, at least some of the current transactions may be used toperform further training of the predictive model. Consequently, thepredictive model may thereby adapt itself to changing conditions.

Either the historical transaction data 1104 or the current transactiondata 1106 might include, according to some embodiments, determinate andindeterminate data. As used herein and in the appended claims,“determinate data” refers to verifiable facts such as the an age of ahome; an automobile type; a policy date or other date; a driver age; atime of day; a day of the week; a geographic location, address or ZIPcode; and a policy number.

As used herein, “indeterminate data” refers to data or other informationthat is not in a predetermined format and/or location in a data recordor data form. Examples of indeterminate data include narrative speech ortext, information in descriptive notes fields and signal characteristicsin audible voice data files.

The determinate data may come from one or more determinate data sources1108 that are included in the computer system 1100 and are coupled tothe data storage module 1102. The determinate data may include “hard”data like a claimant's name, date of birth, social security number,policy number, address, an underwriter decision, etc. One possiblesource of the determinate data may be the insurance company's policydatabase (not separately indicated).

The indeterminate data may originate from one or more indeterminate datasources 1110, and may be extracted from raw files or the like by one ormore indeterminate data capture modules 1112. Both the indeterminatedata source(s) 1110 and the indeterminate data capture module(s) 1112may be included in the computer system 1100 and coupled directly orindirectly to the data storage module 1102. Examples of theindeterminate data source(s) 1110 may include data storage facilitiesfor document images, for text files, and digitized recorded voice files.Examples of the indeterminate data capture module(s) 1112 may includeone or more optical character readers, a speech recognition device(i.e., speech-to-text conversion), a computer or computers programmed toperform natural language processing, a computer or computers programmedto identify and extract information from narrative text files, acomputer or computers programmed to detect key words in text files, anda computer or computers programmed to detect indeterminate dataregarding an individual.

The computer system 1100 also may include a computer processor 1114. Thecomputer processor 1114 may include one or more conventionalmicroprocessors and may operate to execute programmed instructions toprovide functionality as described herein. Among other functions, thecomputer processor 1114 may store and retrieve historical insurancetransaction data 1104 and current transaction data 1106 in and from thedata storage module 1102. Thus the computer processor 1114 may becoupled to the data storage module 1102.

The computer system 1100 may further include a program memory 1116 thatis coupled to the computer processor 1114. The program memory 1116 mayinclude one or more fixed storage devices, such as one or more hard diskdrives, and one or more volatile storage devices, such as RAM devices.The program memory 1116 may be at least partially integrated with thedata storage module 1102. The program memory 1116 may store one or moreapplication programs, an operating system, device drivers, etc., all ofwhich may contain program instruction steps for execution by thecomputer processor 1114.

The computer system 1100 further includes a predictive model component1118. In certain practical embodiments of the computer system 1100, thepredictive model component 1118 may effectively be implemented via thecomputer processor 1114, one or more application programs stored in theprogram memory 1116, and computer stored as a result of trainingoperations based on the historical transaction data 1104 (and possiblyalso data received from a third party). In some embodiments, dataarising from model training may be stored in the data storage module1102, or in a separate computer store (not separately shown). A functionof the predictive model component 1118 may be to determine appropriaterisk parameters and/or scoring algorithms. The predictive modelcomponent may be directly or indirectly coupled to the data storagemodule 1102.

The predictive model component 1118 may operate generally in accordancewith conventional principles for predictive models, except, as notedherein, for at least some of the types of data to which the predictivemodel component is applied. Those who are skilled in the art aregenerally familiar with programming of predictive models. It is withinthe abilities of those who are skilled in the art, if guided by theteachings of this disclosure, to program a predictive model to operateas described herein.

Still further, the computer system 1100 includes a model trainingcomponent 1120. The model training component 1120 may be coupled to thecomputer processor 1114 (directly or indirectly) and may have thefunction of training the predictive model component 1118 based on thehistorical transaction data 1104 and/or information about potentialinsureds. (As will be understood from previous discussion, the modeltraining component 1120 may further train the predictive model component1118 as further relevant data becomes available.) The model trainingcomponent 1120 may be embodied at least in part by the computerprocessor 1114 and one or more application programs stored in theprogram memory 1116. Thus, the training of the predictive modelcomponent 1118 by the model training component 1120 may occur inaccordance with program instructions stored in the program memory 1116and executed by the computer processor 1114.

In addition, the computer system 1100 may include an output device 1122.The output device 1122 may be coupled to the computer processor 1114. Afunction of the output device 1122 may be to provide an output that isindicative of (as determined by the trained predictive model component1118) particular performance metrics and/or evaluation results. Theoutput may be generated by the computer processor 1114 in accordancewith program instructions stored in the program memory 1116 and executedby the computer processor 1114. More specifically, the output may begenerated by the computer processor 1114 in response to applying thedata for the current simulation to the trained predictive modelcomponent 1118. The output may, for example, be a numerical estimateand/or likelihood within a predetermined range of numbers. In someembodiments, the output device may be implemented by a suitable programor program module executed by the computer processor 1114 in response tooperation of the predictive model component 1118.

Still further, the computer system 1100 may include a risk score toolmodule 1124. The risk score tool module 1124 may be implemented in someembodiments by a software module executed by the computer processor1114. The risk score tool module 1124 may have the function of renderinga portion of the display on the output device 1122. Thus, the risk scoretool module 1124 may be coupled, at least functionally, to the outputdevice 1122. In some embodiments, for example, the risk score toolmodule 1124 may direct workflow by referring, to an administrator 1128via risk score platform 1226, current risk score results generated bythe predictive model component 1118 and found to be associated withvarious insurance accounts in an insurer's book of business. In someembodiments, these recommendations may be provided to an administrator1128 who may also be tasked with determining whether or not the resultsmay be improved (e.g., by having a risk mitigation team visit a factoryassociated with a particularly risky account).

Thus, embodiments may provide an automated and efficient way to providea risk scoring system for an insurer's book of business. Consider, forexample, a review of an automobile insurance book of business thatincludes large losses totaling millions of dollars from accidents thatoccurred in a certain time period. A review of unstructured dataimbedded in underwriting documents might have identified a certainpercentage of the policies that should have not been written based on apoor loss history, poor driver qualities, high hazard automobiles, alack of a safety program, etc. The potentially avoided losses might beworth several million dollars, while the premiums were valued at aportion of that amount—resulting in a potential net income of avoidedlosses minus the premium value (representing a certain loss peraccident).

Or consider a book of business having many large automobile insurancelosses each year and, upon further review, it is found that a certainpercentage of those losses could be avoided with heightened scrutiny. Inthis example, the net income loss time the number of claims per yearcould be millions of dollars for the automobile book of business.

As another example, consider a workers' compensation book of businessthat experiences hundreds of large losses per year, each having anaverage value of several hundred thousand dollars. Note that thetendency for workers'compensation large losses is to escalate over timeas medical care matures requiring significant portions of the IncurredBut Not Reported (“IBNR”) to be allocated to these claims. For thepurposes of analysis, assume that the net income ratio is similar tothat of the automobile book of business previously described. The netincome per claim will then be some portion of the average value of largelosses. Assuming that risk score results can help avoid theinappropriate writing of a certain percentage of these claims, theinsurer may have an opportunity to save millions of dollars per year.

According to some embodiments, a combined opportunity of an automobileand workers' compensation initiatives may represent tens of millions ofdollars plus the IBNR load for the workers' compensation line. Althougha review of cases may result in some high risk accounts (that would nothave actually generated a large loss) being removed from the books, thisis likely offset by the IBNR issue. The major remaining variable is theability of an automated tool to identify high potential severity risks.Assuming the automated risk model can help identify half of thepotential accounts, the proposed benefit of the model might comprisehalf of the combined opportunity of initiatives. If this approach can beapplied to the property and general liability insurance, the opportunitymay be larger. This approach may also be leveraged across segments andpotentially to group benefits and personal lines.

Note that, in general, a certain percentage of reviewed accounts fail ona key risk/reward evaluation conducted during a quality review. Therisk/reward review may comprise, for example, an evaluation of whetherthe account provides enough premium for the potential risk. The mostcommon reason for an account to miss the mark on a risk/reward analysisis a lack of documentation. Lack of documentation presents materialconcerns because it may indicate an insufficient investigation.Moreover, risk characteristics with significant potential to generatesevere losses might not be uncovered during a cursory investigation. Itis contemplated that some risks will be marked as fine, some asquestionable, and some as unacceptable at any price by the system.

Embodiments described herein may provide a system that can evaluateevery open account, on a regular basis, to determine which accounts aremost likely to generate significant losses. Ideally, accounts that arescored can also be used to improve a quality review process by aninsurer's home office and/or field staff. Such an approach maypositively impact the risk profile of a book of business letting thestaff spend additional time working on more complicated files.

According to some embodiments, each account in the insurer's book ofbusiness will receive a score and thresholds may automatically triggerreview by underwriting and quality staff. Files that score with thehighest likelihood of generating a significant loss may be reviewed andactions will be taken as appropriate. Note that rules generated by thesystem might be combined with business rules. Using these in combinationmay create an opportunity to handle both high and low potential severityaccounts. High potential severity accounts may be referred to anunderwriting unit responsible including the home office referralunderwriter. Additionally, referrals may be provided to a quality teamfor review. Low potential severity account lists may be provided tounderwriting staff in an effort to improve efficiency (e.g., by notoverworking relatively low-risk accounts). The rules generated by themodel can also help to assign account level risk profiles by line ofbusiness (workers' compensation, automobile insurance, etc.). Theseprofiles will enable reserving to better evaluate the risk load for thebook of business, as well as guide underwriting and the development ofpremium indications both in a home office and the field.

According to some embodiments, a quality team uses a report onstructured data to help select the most appropriate accounts for review.The report may contain a variety of structured data elements that canhelp to identify drivers of severity.

Another source of relevant underwriting information that may help topredict severity resides in a risk evaluation document. The riskevaluation document may be a required document generally completedbefore or shortly after binding a policy. The information present in therisk evaluation document may be unstructured and require text miningapplications to identify triggers or flags in the documentation.

As another example, a risk engineering survey may represent acomprehensive survey on a subset of risks as requested by underwritingstaff. Note that a greater amount of detail may be present in thesurveys as compared to what is ultimately documented in the riskevaluation document. From a large loss review process, it may beestimated that 33% loss control had open recommendations for thesepolicies. When this finding was compared with other policies withsmaller business insurance losses, the open recommendations in the losscontrol report were twice as frequent (33% as compared to 16%). Openrecommendations in risk engineering surveys may present a significantopportunity to identify potential severity.

As still another example, claims notes may contain information likelythat will help to identify unsuitable risks (e.g., that have a claimhistory). Note that claims notes may typically be stored in a minabledatabase.

As yet another example, web “crawling” technology exists that can obtaintext from websites that might raise underwriting concerns. When thisprocess is automated, it may serve as a significant competitiveadvantage due to the incorporation of useful data that does notcurrently exist in underwriter notes.

As another approach, a system might use a cloud-based advanced dataanalytics platform to conduct a Natural Language Process (“NLP”)analysis to identify accounts that are likely to result in a failinggrade for the risk/reward analysis.

Thus, embodiments may help identify potential large loss claim accountswith unacceptable risk characteristics. For example, a scoring tool mayautomatically identify accounts with high potential severity usinginternal data, external data, text mining, etc. Some embodiments may beassociated with an underwriting application and/or a quality/referralapplication that provides continuous monitoring and management of bookquality to reduce unacceptable risks.

An underwriting application may, according to some embodiments, comprisea front-end application that provides premium indications and flags tohelp reduce underwriting time and help ensure consistency (similarresearch will be performed during risk analysis). The underwritingapplication may provide benchmark information by providing risk profileinformation as compared to similar accounts. Some of the informationexamined by any of the embodiments described herein might include, forexample, one or more of a company profile, an industry code, an employeeor employer address, a measurement of litigiousness, financialinformation, average wage information, credit report information,actuarial data, an audio file of a Chief Executive Officer's commentsmade during a teleconference with investors (e.g., in accordance toautomated speech-to-text processing), data from social media cites(e.g., may employees are discussing potential layoffs), etc. Note thatdifferent types of information may be combined (e.g., after being givendifferent weights as appropriate) to improve the accuracy of the system.

A quality/referral application may provide a book-of-business view usinga risk score model tuned to frequency, premium indication, and/orstructured data. The quality/referral application may predict alikelihood of a large loss using a proactive and repeatable method toexamine structured and unstructured data. FIG. 12 is an example of abook of business view 1200 according to some embodiments. In particular,a book of business 1210 may include a number of separate accounts 1212that may be reviewed and assigned to a risk category 1220 such as: (i) alow large loss risk potential category representing the safest 80% ofaccounts, (ii) a high large loss risk potential category representingthe least safe 5% of accounts, and (iii) a moderate large loss riskpotential category representing the remaining accounts.

According to some embodiments, external open data and/or restricted datamay be evaluated. Open data might comprise, for example, informationwith no restrictions on usage, such as newspapers, municipal web sites,insured web sites, governmental agencies, etc. Restricted data mightinclude, for example, social networking web sites, customer review websites, indexes of communication addresses, etc. FIG. 13 illustrates anew variable proving ground 1300 in accordance with some embodiments.The proving ground 1300 may input text mining internal data 1310,structured external data 1320, and text mining external data 1330.Moreover a scoring matrix 1350 may rate data in terms of input frequency(high or low) and predictive power (high or low) and provide an outputto a risk score 1360 and/or a premium indication portal 1370. Thescoring matrix 1350, according to some embodiments, may categorize: (i)high frequency, low predictive power data as being not particularlyrelevant, (ii) low frequency, low predictive power data as recommendingthat underwriting staff be cautious, (iii) high frequency, highpredictive power data as recommending that guide adjustment bediscontinued, and (iv) low frequency, high predictive power data asneeding underwriter discretion (e.g., a manager's approval might berequired).

According to some embodiments, the system may be continuously monitoringand rating accounts. Consider, for example, a workers'compensationbook-of-business that includes a number of painting contractors.Moreover, an insurer might not have an appetite to insure contractorswho spend a substantial amount of time on ladders working on abuilding's exterior (e.g., because of the potential large losses thatmight be associated with an employee falling off of a ladder). FIG. 14is associated with a large workers' compensation loss on anout-of-appetite risk 1400 according to some embodiments. In thisexample, a rating 1410 web site might be automatically accessed by thesystem. Moreover, text-based details 1420 about a particular contractormight be analyzed looking for particular key words and/or phrases. Asillustrated in FIG. 1420, the underlined text might represent words andphrases that cause the system to automatically raise a risk flag.Similarly, the system might automatically access and analysis customerreviews 1430 submitted about the contractor. Note that other informationmay also be analyzed by the system. For example, drone or satelliteinformation might be used to determine a type of roof associated with aparticular customer, text mining might be performed on safety inspectionreports, claim handler notes, etc.

Note that the system might review accounts on a periodic basis (e.g.,once per week) or in substantially real time (e.g., the system mightcontinuously look for, and analyze, information about accounts). FIG. 15illustrates a periodic execution of a risk review implementation 1500according to some embodiments. In this example, a company might beautomatically identified as a high risk at 1510. This mightautomatically result in a referral of the account (e.g., to a riskmanager). The account may be reviewed at 1520 and, if appropriate,actions may be taken at 1530 (e.g., non-renewal of an account, riskengineering recommendations) until the account is closed at 1540. Ifsuch steps are not appropriate (e.g., the originally raised flag were,upon further inspection, found to be harmless), the flags may be clearedand the account may continue to be monitored until new risks areautomatically identified at 1550.

The following illustrates various additional embodiments of theinvention. These do not constitute a definition of all possibleembodiments, and those skilled in the art will understand that thepresent invention is applicable to many other embodiments. Further,although the following embodiments are briefly described for clarity,those skilled in the art will understand how to make any changes, ifnecessary, to the above-described apparatus and methods to accommodatethese and other embodiments and applications.

Although specific hardware and data configurations have been describedherein, note that any number of other configurations may be provided inaccordance with embodiments of the present invention (e.g., some of theinformation associated with the displays described herein might beimplemented as a virtual or augmented reality display and/or thedatabases described herein may be combined or stored in externalsystems). Moreover, although embodiments have been described withrespect to particular types of insurance policies, embodiments mayinstead be associated with other types of insurance. Still further, thedisplays and devices illustrated herein are only provided as examples,and embodiments may be associated with any other types of userinterfaces. For example, FIG. 16 illustrates a handheld risk scoredashboard display 1600 according to some embodiments. The dashboarddisplay 1600 might include graphical icons providing information about abook of business based on geographic location 1610 and/or an executivedashboard area displaying information by industry 1620. According tosome embodiments, the display 1600 might let a user drill down to theaccount, state, industry, office, or underwrite level to betterunderstand a book of business. In some embodiments, an entire book ofbusiness might be assigned an overall riskiness score, and user mightaccess the display 1600 to determine what that score means.

According to some embodiments, risk score results might be used byauditors, underwrites, risk engineers, etc. The risk core results mightbe used, for example, to help determining an appropriate premium,identify fraud, make renewal decisions, identify a need for a manual,telephonic or paper audit, assign risk engineering resources, influencebroker/agent interactions and commission arrangements, adjust risk modelparameters and assumptions, recommend deductibles and/or policy limits,make proactive offers (e.g., recommended insurance riders orendorsements to tailor coverage as appropriate), etc. Consider, forexample, FIG. 17 illustrates a potential response display 1700 thatmight be presented in connection with a high risk score account in aworkers' compensation book of business. The display 1700 might includeseveral user-selectable options 1710 to respond the account's high levelof risk. The options 1710 might include, for example, referring theaccount to a manager, modifying coverage limits, altering a renewaldecision, etc. According to some embodiments, a set of available options1710 might be automatically determined by the system as appropriate(e.g., as represented by the non-appropriate options being displayed as“grayed out” in FIG. 17).

The present invention has been described in terms of several embodimentssolely for the purpose of illustration. Persons skilled in the art willrecognize from this description that the invention is not limited to theembodiments described, but may be practiced with modifications andalterations limited only by the spirit and scope of the appended claims.

What is claimed:
 1. A system to evaluate relative risks based at leastin part on source inputs received via a distributed communicationnetwork by an automated back-end application computer server,comprising: (a) a structured data element information source input; (b)an unstructured data element information source input; (c) an external,third-party data element information source input; (d) a communicationport to facilitate an exchange of electronic messages with thestructured data element information source input, the unstructured dataelement information source input, and the external, third-party dataelement information source input via the distributed communicationnetwork; (e) a data store containing electronic files associated with aset of entities; and (f) the back-end application computer server,coupled to the communication port and the data store, programmed to: (i)for each of a plurality of the entities in the set of entities, accessthe electronic files in the data store associated with that entity, (ii)retrieve structured data elements for that entity from the structureddata element information source input, (iii) retrieve unstructured dataelements for that entity from the unstructured data element informationsource input, (iv) retrieve external, third-party data elements for thatentity from the external, third-party data element information sourceinput, (v) determine at least one variable associated with an automatedrisk model, the at least one variable being assigned a predictivequality based at least in part on an input frequency over time of thevariable, using a variable proving ground tool comprising a scoringmatrix configured to rate variables based on at least input frequencyover time, (vi) execute the automated risk model to assign a risk scoreto that entity based on the at least one variable, the electronic files,the structured data elements, the unstructured data elements, and theexternal, third-party data elements for that entity, and (vii) transmitindications of the risk scores for the plurality of entities.
 2. Thesystem of claim 1, wherein the data store is associated with aninsurer's book of business, and each entity in the data store comprisesan insured party.
 3. The system of claim 2, wherein the risk scorerepresents a likelihood of a substantial potential loss for the insurer.4. The system of claim 3, wherein at least some of the insured partiesare associated with at least one of: workers' compensation insurance,commercial automobile insurance, general liability insurance, propertyinsurance, property casualty insurance, homeowners' insurance, personallines of insurance, group benefits insurance, and life insurance.
 5. Thesystem of claim 3, wherein the automated risk model is executed at leastone of: when initiated by an administrator, responsive to a specificquery submitted by the administrator, on a periodic basis, on a dailybasis, on a weekly basis, on a monthly basis, and on a substantiallyconstant basis.
 6. The system of claim 3, wherein the risk scorecomprises at least one of: a numerical value for each insured party, anda ranked list of insured parties.
 7. The system of claim 6, wherein theplurality of insured parties are insured parties that that do notsatisfy a pre-determined threshold criteria indicating that asubstantial potential loss is unlikely.
 8. The system of claim 3,wherein the structured data elements are associated with at least oneof: internally stored data elements about the insured party, andgovernmental or regulatory data elements.
 9. The system of claim 3,wherein the unstructured data elements are associated with at least oneof: web pages of the insured party, web pages mentioning the insuredparty, text information, image information, satellite image information,drone image information, video information, and audio information. 10.The system of claim 3, wherein the external, third-party data elementsare associated with at least one of social media content and online,user-submitted review content.
 11. A computerized method to evaluaterelative risks based at least in part on source inputs received via adistributed communication network by an automated back-end applicationcomputer server, comprising: accessing a data store containingelectronic files associated with a set of entities to retrieve, for eachof a plurality of the entities in the set of entities, electronic filesassociated with that entity; retrieving structured data elements forthat entity from a structured data element information source input;retrieving unstructured data elements for that entity from anunstructured data element information source input; retrieving external,third-party data elements for that entity from an external, third-partydata element information source input; determining at least one variableassociated with an automated risk model, the at least one variable beingassigned a predictive quality based at least in part on an inputfrequency over time of the variable, using a variable proving groundtool comprising a scoring matrix configured to rate variables based onat least input frequency over time; executing, by a computer processorof the back-end application computer server, the automated risk model toassign a risk score to that entity based on the at least one variable,the electronic files, the structured data elements, the unstructureddata elements, and the external, third-party data elements for thatentity; and transmitting indications of the risk scores for theplurality of entities.
 12. The method of claim 11, wherein the datastore is associated with an insurer's book of business, and each entityin the data store comprises an insured party.
 13. The method of claim12, wherein the risk score represents a likelihood of a substantialpotential loss for the insurer.
 14. The method of claim 13, wherein atleast some of the insured parties are associated with at least one of:workers' compensation insurance, commercial automobile insurance,general liability insurance, property insurance, and property casualtyinsurance.
 15. The method of claim 13, wherein the automated risk modelis executed at least one of: when initiated by an administrator, on aperiodic basis, on a daily basis, on a weekly basis, and on a monthlybasis.
 16. The method of claim 13, wherein the risk score comprises atleast one of: a numerical value for each insured party, and a rankedlist of insured parties.
 17. The method of claim 13, wherein: thestructured data elements are associated with at least one of: internallystored data elements about the insured party, and governmental orregulatory data elements, the unstructured data elements are associatedwith at least one of: web pages of the insured party, web pagesmentioning the insured party, text information, image information,satellite image information, drone image information, video information,and audio information, and the external, third-party data elements areassociated with at least one of social media content and online,user-submitted review content.
 18. A non-transitory, computer-readablemedium storing instructions, that, when executed by a processor, causethe processor to perform a method to evaluate relative risks based atleast in part on source inputs received via a distributed communicationnetwork by an automated back-end application computer server, the methodcomprising: accessing a data store containing electronic filesassociated with a set of entities to retrieve, for each of a pluralityof the entities in the set of entities, electronic files associated withthat entity; retrieving structured data elements for that entity from astructured data element information source input; retrievingunstructured data elements for that entity from an unstructured dataelement information source input; retrieving external, third-party dataelements for that entity from the external, third-party data elementinformation source input; determining at least one variable associatedwith an automated risk model, the at least one variable being assigned apredictive quality based at least in part on an input frequency overtime of the variable, using a variable proving ground tool comprising ascoring matrix configured to rate variables based on at least inputfrequency over time; executing, by a computer processor of the back-endapplication computer server, the automated risk model to assign a riskscore to that entity based on the at least one risk factor, theelectronic files, the structured data elements, the unstructured dataelements, and the external, third-party data elements for that entity;transmitting indications of the risk scores for the plurality ofentities; automatically determining a set of potential responses for aselected entity based on the associated risk score; and presenting agraphical user interface display including the automatically determinedset of potential responses.
 19. The medium of claim 18, wherein the datastore is associated with an insurer's book of business, and each entityin the data store comprises an insured party.
 20. The medium of claim19, wherein the risk score represents a likelihood of a substantialpotential loss for the insurer.
 21. The medium of claim 20, wherein atleast some of the insured parties are associated with at least one of:workers' compensation insurance, commercial automobile insurance,general liability insurance, property insurance, and property casualtyinsurance.
 22. The medium of claim 20, wherein the automated risk modelis executed at least one of: when initiated by an administrator, on aperiodic basis, on a daily basis, on a weekly basis, and on a monthlybasis.
 23. The medium of claim 20, wherein the risk score comprises atleast one of: (i) a numerical value for each insured party, and (ii) aranked list of insured parties.
 24. The medium of claim 20, wherein: thestructured data elements are associated with at least one of: Internallystored data elements about the insured party, and governmental orregulatory data elements, the unstructured data elements are associatedwith at least one of: web pages of the insured party, web pagesmentioning the insured party, text information, image information,satellite image information, drone image information, video information,and audio information, and the external, third-party data elements areassociated with at least one of social media content and online,user-submitted review content.